Medical device visualization

ABSTRACT

Systems and methods present multiple images of a three-dimensional model, including a three-dimensional representation of an anatomic structure and a representation of a medical device relative to the anatomic structure, on a graphical user interface. The multiple images correspond to different views (e.g., in multiple, different planes) of the three-dimensional model to provide complementary spatial information regarding a position of the medical device relative to the anatomic structure during a medical procedure performed on the anatomic structure. Such complementary spatial information can, for example, provide spatial context to facilitate controlling the position of the medical device relative to the anatomic structure during the medical procedure.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. §119(e) of U.S.Prov. App. No. 62/330,910, filed May 3, 2016, U.S. Prov. App. No.62/337,541, filed May 17, 2016, U.S. Prov. App. No. 62/338,068, filedMay 18, 2016, U.S. Prov. App. No. 62/357,600, filed Jul. 1, 2016, U.S.Prov. App. No. 62/367,763, filed Jul. 28, 2016, with the entire contentsof each of these applications hereby incorporated herein by reference.

This application is also related to the commonly-owned U.S. patentapplication filed on even date herewith and having Attorney DocketNumber AFRA-0003-P01 and entitled “ANATOMICAL MODEL DISPLAYING,” theentire contents of which are hereby incorporated herein by reference.

BACKGROUND

Three-dimensional models are used to assist in the placement or use of adevice when such placement or use is not easily observable or practical.For example, in medical procedures, three-dimensional models are used toassist in the placement and use of medical devices for the diagnosis ortreatment of patients. An example of such a medical procedure carriedout with the assistance of a three-dimensional model is the use of acatheter to deliver radio frequency (“RF”) ablation to form lesions thatinterrupt abnormal conduction in cardiac tissue, thus terminatingcertain arrhythmias in the heart.

SUMMARY

The present disclosure is directed to devices, systems, and methods ofpresenting multiple images of a three-dimensional model on a graphicaluser interface to facilitate visualizing the position of a medicaldevice relative to an anatomic structure during a medical procedure. Themultiple images can correspond to different views (e.g., in multiple,different planes) of the three-dimensional model to providecomplementary spatial information regarding the position of the medicaldevice relative to the anatomic structure during a medical procedureperformed on the anatomic structure. Such complementary spatialinformation can, for example, provide spatial context for controllingthe position of the medical device relative to the anatomic structureduring the medical procedure. Additionally, or alternatively, one ormore of the images can depict a fractional view of the three-dimensionalmodel such that internal and external surfaces of the three-dimensionalmodel are observable on a single graphical user interface as the medicaldevice is moved relative to the anatomic structure during a medicalprocedure. Thus, for example, the systems and methods of the presentdisclosure can address visualization challenges associated with certainmedical procedures (e.g., cardiac procedures) in which it can bedesirable to make observations from a perspective looking into ananatomic structure within which a medical device is positioned.

According to one aspect, a method includes receiving a signal indicativeof a medical device in an anatomic structure of a patient, constructinga three-dimensional model including a three-dimensional representationof the anatomic structure and a representation of the medical devicerelative to the anatomic structure based at least in part on thereceived location signal, clipping the three-dimensional model based atleast in part on the received location signal, and displaying, on agraphical user interface, a first image and a second image. Clipping thethree-dimensional model includes removing a first portion of thethree-dimensional model relative to a first clipping surfaceintersecting the three-dimensional model to form a clipped model. Thefirst image includes a projection of the three-dimensional model on afirst viewing window of a first image plane, and the second imageincludes a projection of the clipped model on a second viewing window ofa second image plane.

In some implementations, the second image plane can intersect the firstimage plane.

In certain implementations, the method can further include updating thefirst image and the second image based on at least one of a user inputand a received location signal of the medical device.

In some implementations, the second image plane can be in a fixedorientation relative to the first image plane.

In certain implementations, the three-dimensional representation of theanatomic structure can include a boundary surface and displaying thefirst image can include displaying a portion of the boundary surfacethat is not shown in the second image.

In some implementations, clipping the three-dimensional model canfurther include removing a second portion of the three-dimensional modelrelative to a second clipping surface, and the clipped model can besubstantially between the first clipping surface and the second clippingsurface. For example, at least a portion of the first clipping surfaceand the second clipping surface are substantially parallel to oneanother.

In certain implementations, the first clipping surface can be a plane.For example, the first clipping surface can be substantially parallel tothe second image plane.

In some implementations, constructing the three-dimensional model caninclude receiving one or more images of the anatomic structure andregistering the images to a coordinate system of a sensor providing thesignal indicative of the location of the medical device.

In some implementations, constructing the three-dimensional model caninclude updating the representation of the medical device based on thereceived location signal. For example, the first clipping surface canextend through the updated representation of the medical device.

In certain implementations, the received location signal can be atime-varying signal and clipping the three-dimensional model can furtherinclude processing the time-varying received location signal andselecting the first clipping surface based on the processed, receivedlocation signal. For example, processing the time-varying receivedlocation signal can include low-pass filtering the received locationsignal.

In certain implementations, the second image can include a projection ofthe clipped model extending in a direction from the second viewingwindow, toward the three-dimensional model, and substantially orthogonalto the second image plane.

In some implementations, the three-dimensional model can include aboundary surface of the anatomic structure and a portion of the boundarysurface in the clipped model in the second image is more translucentthan a corresponding portion of the boundary surface in thethree-dimensional model in the first image.

In certain implementations, the three-dimensional model can include aboundary surface of the anatomic structure and displaying the secondimage can include highlighting a contour of the boundary surface withina predetermined distance of the first clipping surface. For example,highlighting the contour of the boundary surface within thepredetermined distance of the first clipping surface includeshighlighting a contour of the boundary surface intersected by the firstclipping surface.

In some implementations, displaying the first image and the second imagecan include displaying the first image and the second imagesimultaneously on the graphical user interface.

In certain implementations, an included angle between the second imageplane and the first image plane is less than or equal to 90 degrees andgreater than about 60 degrees. For example, the second image plane canbe orthogonal to the first image plane.

In some implementations, the method can further include adjusting thefirst image plane and maintaining the second image plane in a fixedorientation relative to the first image plane. Additionally, oralternatively, the second image plane can be restricted to a directionsuperior to the representation of the location of the medical device inthe three-dimensional model. Further, or instead, adjusting the firstimage plane can include orienting the first image plane relative to thelocation, in the anatomic structure of the patient, of the medicaldevice.

In certain implementations, displaying the first image and the secondimage on the graphical user interface can include displaying the firstimage as larger than the second image.

In some implementations, displaying the first image and the second imageon the graphical user interface can include adjusting the size of thethree-dimensional model as projected onto one or both of the firstviewing window and the second viewing window.

In certain implementations, displaying the first image and the secondimage on the graphical user interface can include adjusting a distancefrom at least one of the first image plane or the second image plane tothe three-dimensional model.

In some implementations, displaying the first image and the second imagecan include sizing at least one of the first viewing window or thesecond viewing window. For example, sizing the first viewing window canbe based on at least one dimension of the three-dimensional model in thefirst image plane. Additionally, or alternatively, sizing the secondviewing window can be based on a bounding volume defined around thethree-dimensional model. As an example, sizing the second viewing windowcan include sizing the second viewing window based on a dimension (e.g.,a maximum dimension) of the bounding volume. Further, or instead, thebounding volume can be a sphere (e.g., a sphere having a diameter basedon a maximum dimension of the three-dimensional model).

In certain implementations, the method can further include receiving asignal indicative of a location of a treatment applied by the medicaldevice to the anatomic structure. In certain instances, constructing thethree-dimensional model can include adding visual indicia to thethree-dimensional model, the visual indicia corresponding to thelocation of the treatment, and at least one of the first image and thesecond image including the visual indicia.

In some implementations, the method can further include receiving asignal indicative of a location of an anatomic feature of the anatomicstructure. In some instances, constructing the three-dimensional modelcan include adding visual indicia to the three-dimensional model, thevisual indicia corresponding to the location of the anatomic feature,and the first image and the second image each including the visualindicia.

In certain implementations, the second image can include visual indiciahighlighting a boundary of the medical device. For example, the visualindicia highlighting the boundary of the medical device can vary incolor according to one or more of time and around the boundary of themedical device.

According to another aspect, a non-transitory, computer-readable storagemedium has stored thereon computer executable instructions for causingone or more processors to receive a signal indicative of a location of amedical device in an anatomic structure of a patient, construct athree-dimensional model including a three-dimensional representation ofthe anatomic structure and a representation of the medical devicerelative to the anatomic structure based at least in part on thereceived location signal, clip the three-dimensional model based atleast in part on the received location signal, and display, on agraphical user interface, a first image and a second image. Clipping thethree-dimensional model includes removing a first portion of thethree-dimensional model relative to a first clipping surfaceintersecting the three-dimensional model to form a clipped model. Thefirst image includes a projection of the three-dimensional model on afirst viewing window of a first image plane, and the second imageincludes a projection of the clipped model on a second viewing window ofa second image plane.

According to yet another aspect, a method includes receiving a signalindicative of a location of a medical device in an anatomic structure ofa patient, updating a three-dimensional model based at least in part onthe received location signal, forming a first image including aprojection (on a first viewing window of a first image plane)of at leastone portion of the updated three-dimensional model, forming (on a secondviewing window of a second image plane) a second image of anotherportion of the updated three-dimensional model, and displaying, on agraphical user interface, the first image and the second image. Theportion of the three-dimensional model projected on the second viewingwindow is less than the at least one portion of the updatedthree-dimensional model projected on the first viewing window, and thesecond image plane intersects the first image plane. Thethree-dimensional model includes a three-dimensional representation ofthe anatomic structure and a representation of the medical devicerelative to the anatomic structure.

In some implementations, the method further includes adjusting the firstimage plane and maintaining the second image plane in a fixedorientation relative to the first image plane. For example, adjustingthe first image plane can include orienting the first image planerelative to the received location signal. Additionally, oralternatively, orienting the first image plane relative to the receivedlocation signal can include orienting the first image plane in adirection perpendicular to a weighted sum of surface normal vectorsalong a portion, closest to the medical device, of a boundary surface ofthe anatomic structure in the three-dimensional model.

In certain implementations, displaying the first image and the secondimage can include displaying the first image and the second imagesimultaneously on the graphical user interface.

In some implementations, the second image plane can be orthogonal to thefirst image plane.

According to still another aspect, a system includes a catheter and acatheter interface unit. The catheter has a distal portion and aproximal portion. The distal portion is mechanically coupled to theproximal portion, and the distal portion is insertable into a chamber ofa patient's heart and movable within the chamber via manipulation of theproximal portion. The catheter interface unit is in electricalcommunication with the catheter and includes a graphical user interface,one or more processors, and a non-transitory, machine-readable storagemedium having stored thereon machine executable instructions for causingthe one or more processors to receive a signal indicative of a locationof the distal portion of the catheter in the chamber of the patient'sheart, based at least in part on the received location signal, update athree-dimensional model, the three-dimensional model including athree-dimensional representation of the heart chamber and arepresentation of the catheter relative to the heart chamber, anddisplay a first image and a second image on the graphical userinterface, wherein the first image includes a projection of the updatedthree-dimensional model to a first viewing window of a first imageplane, and the second image includes a projection of a portion of theupdated three-dimensional model on a second viewing window of a secondimage plane, the second image plane intersecting the first image plane,and the portion of the updated three-dimensional model displayed in thesecond viewing window being less than the entirety of thethree-dimensional model displayed on the first viewing window.

In certain implementations, the second image plane can have a fixedorientation relative to the first image plane. For example, the secondimage plane can be orthogonal to the first image plane.

In some implementations, the instructions to display the first image caninclude instructions to adjust the first image plane based on thereceived location signal. For example, the instructions to adjust thefirst image plane can include instructions to adjust the first imageplane to an orientation perpendicular to a weighted sum of surfacenormal vectors along a portion, closest to the distal portion of thecatheter, of a boundary surface of the chamber of the heart in thethree-dimensional model.

Implementations can include one or more of the following advantages.

In certain implementations, displaying a first image and a second imageon the graphical user interface can include displaying a projection ofthe three-dimensional model on a first viewing window of the first imageplane and displaying the projection of only a determined portion of thethree-dimensional model on a second viewing window of the second imageplane. It should be appreciated that displaying the three-dimensionalmodel in one perspective and a displaying only a portion of thethree-dimensional model in another perspective can provide complementaryviews of the medical device that can be useful for maneuvering themedical device in three-dimensions. Additionally, or alternatively, suchmultiple perspectives can facilitate visualization from a perspectivelooking into a surface of an anatomic structure within which the medicaldevice is positioned while providing a perspective useful for navigationof the medical device relative to the surface of the anatomic structure.

In some implementations, a second image plane can extend through arepresentation of the medical device and the determined portion of thethree-dimensional model can extend in a direction away from the secondimage plane. In such implementations, a corresponding second image canbe unobstructed by the three-dimensional representation of the anatomicstructure, thus facilitating visualization of the medical devicerelative to a surface of the anatomic structure.

In certain implementations, a first image and a second image can beupdated based on a received signal indicative of a location of a medicaldevice in an anatomic structure. Accordingly, the first image and thesecond image can provide the physician with dynamically changinginformation about the position of the medical device relative to theanatomic structure during the medical procedure. Thus, for example, thefirst image and the second image can be useful for providing thephysician with substantially real-time guidance for maneuvering themedical device during the medical procedure.

Other aspects, features, and advantages will be apparent from thedescription and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic representation of a system during a medicalprocedure being performed on a patient.

FIG. 2 is a perspective view of a catheter of the system of FIG. 1.

FIG. 3 is a schematic representation of a tip section of the catheter ofFIG. 1 disposed in an anatomic cavity during the medical procedureassociated with FIG. 1.

FIG. 4 is a schematic representation of a graphical user interface ofthe system of FIG. 1, the graphical user interface displayingprojections of a three-dimensional model associated with the medicalprocedure of FIG. 1, and the respective projections shown in a firstviewing window and a second viewing window of the graphical userinterface.

FIG. 5 is a schematic representation of the projections of thethree-dimensional model of FIG. 4 onto the first viewing window and thesecond viewing window of the graphical user interface of FIG. 4.

FIG. 6 is a schematic representation of two clipping surfaces slicingthe three-dimensional model of FIG. 4.

FIG. 7 is a flowchart of an exemplary method of displaying multipleimages of a three-dimensional model on a graphical user interface.

FIG. 8 is a flow chart of another exemplary method of displayingmultiple images of a three-dimensional model on a graphical userinterface.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

The present disclosure is generally directed to devices, systems, andmethods of controlling, on a graphical user interface, multiple displayviews of a medical device relative to an anatomic structure of a patientduring a medical procedure being performed on the anatomic structure ofthe patient. For example, the systems and methods of the presentdisclosure can be used to provide spatial context to a physician tofacilitate multi-directional positioning of the medical device relativeto the anatomic structure during the medical procedure. By way ofnon-limiting example and for the sake of clarity of explanation, certainaspects of the present disclosure are described with respect tovisualization of a cardiac catheter inserted into a heart cavity as partof a diagnostic and/or treatment procedure. However, it should beappreciated that, unless otherwise specified or made clear from thecontext, the systems and methods of the present disclosure can be usedfor any of various different medical procedures in which athree-dimensional model of an anatomic structure (e.g., a hollowanatomic structure) is used to visualize a position of a medical devicein the hollow anatomic structure during the medical procedure. Forexample, the systems and methods of the present disclosure can,additionally or alternatively, be used in interventional pulmonology,brain surgery, and/or sinus surgery (e.g., sinuplasty).

As used herein, the term “physician” can include any type of medicalpersonnel who may be performing or assisting a medical procedure. Theterm “medical procedure” can include any manner and type of medicalprocedure and, therefore, should be considered to include any and allmanner and forms of diagnosis and treatment, unless otherwise specifiedor made clear from the context.

As used herein, the term “patient” should be considered to include anymammal, including a human, upon which a medical procedure is beingperformed.

FIG. 1 is a schematic representation of a system 100 during a medicalprocedure (e.g., an ablation treatment) being performed on a patient102. The system 100 can include a catheter 104 connected via anextension cable 106 to an interface unit 108. The interface unit 108(e.g., a catheter interface unit) can include a processing unit 109(e.g., one or more processors), a graphical user interface 110, and astorage medium 111. The graphical user interface 110 and the storagemedium 111 can be in electrical communication (e.g., wiredcommunication, wireless communication, or both) with the processing unit109.

As described in further detail below, the graphical user interface 110can be used as part of diagnosis and/or treatment of cardiac tissue ofthe patient 102 by, for example, displaying multiple images, with eachimage corresponding to a different perspective of a three-dimensionalmodel during a medical procedure. As compared to systems providing adisplay of only a single image of a three-dimensional model, displayingmultiple, different views of the three-dimensional model on thegraphical user interface 110 according to any one or more of the methodsdescribed herein can provide a physician with improved spatial contextfor three-dimensional movement of the catheter 104 relative to one ormore surfaces of the anatomic structure. As a specific example relatedto an exemplary treatment, displaying multiple, different images of thethree-dimensional model on the graphical user interface 110 according toany one or more of the methods described herein can facilitatethree-dimensional movement of the catheter 104 within the anatomicstructure to create one or more lesions in a desired pattern on one ormore surfaces of the anatomic structure represented by thethree-dimensional model.

As also described in greater detail below, the multiple images displayedon the graphical user interface can be based on a received location ofthe catheter 104 in the heart cavity. Accordingly, the multiple imagescan be updated (e.g., automatically) as the location of the catheter 104changes during a medical procedure. Such dynamic updates to the multipleimages can, in turn, be useful for providing the physician with updatedviews of the three-dimensional model in the vicinity of the catheter104, facilitating both fine and coarse adjustments to the position ofthe catheter 104 relative to a surface of the anatomic structure.

Referring to FIGS. 1 and 2, the catheter 104 can be any of variousdifferent catheters known in the art (e.g., for diagnosis, treatment, orboth). Thus, the catheter 104 can include a handle 120, a catheter shaft122, and a tip section 124. The catheter shaft 122 can include aproximal portion 126 secured to the handle 120, and a distal portion 128coupled to the tip section 124.

The tip section 124 generally includes any portion of the catheter 104that directly or indirectly engages tissue for the purpose of treatment,diagnosis, or both and, therefore, can include all manner and type ofcontact and/or non-contact interaction with tissue known in the art. Forexample, the tip section 124 can include contact and/or non-contactinteraction with tissue in the form of energy interaction (e.g.,electrical energy, ultrasound energy, light energy, and any combinationsthereof) and further, or instead, can include measurement of electricalsignals emanating from tissue. Thus, for example, the tip section 124can deliver energy (e.g., electrical energy) to tissue in the anatomicstructure as part of any number of procedures including treatment,diagnosis, or both.

In certain implementations, the delivery of energy from the tip section124 to the tissue can be through direct contact between the tip section124 and the tissue. In such implementations, it may be particularlydesirable for the graphical user interface 110 to display multiple,different images of the three-dimensional model to provide the physicianwith knowledge of the position of the tip section 124 relative to one ormore surfaces of the anatomic structure. It should be furtherappreciated that the systems and methods of the present disclosure canbe implemented using any number and manner of designs of the catheter104 that rely upon, or at least derive some benefit from, knowledge oflocation of the tip section 124 relative to one or more surfaces of theanatomic structure.

The catheter 104 can further include a magnetic position sensor 130along the distal portion 128 of the catheter shaft 122. The magneticposition sensor 130 can be any of various magnetic position sensors wellknown in the art and can be positioned at any point along the distalportion 128. The magnetic position sensor 130 can, for example, includeone or more coils that detect signals emanating from magnetic fieldgenerators. One or more coils for determining position with five or sixdegrees of freedom can be used. The magnetic field detected by themagnetic position sensor 130 can be used to determine the positionand/or orientation of the distal portion 128 of the catheter shaft 122according to one or more methods commonly known in the art such as, forexample, methods based on using a magnetic sensor, such as the magneticposition sensor 130, to sense magnetic fields and using a look-up tableto determine location of the magnetic position sensor 130. Accordingly,because the tip section 124 is coupled to the distal portion 128 of thecatheter shaft 122 in a known, fixed relationship to the magneticposition sensor 130, the magnetic position sensor 130 can also providethe location of the tip section 124. While the location of the tipsection 124 is described as being determined based on magnetic positionsensing, other position sensing methods can additionally oralternatively be used. For example, the location of the tip section 124can be additionally, or alternatively, based on impedance, ultrasound,and/or imaging (e.g., real time MRI or fluoroscopy).

Referring to FIGS. 1-5, a three-dimensional representation 134 of ananatomic cavity 132 (e.g., an anatomic structure such as a heart cavity)of the patient 102 can be built based on known positions of the tipsection 124 of the catheter 104 in the anatomic cavity 132 (e.g., priorto application of an ablation treatment or other type of treatment) andadditionally, or alternatively, based on images of the anatomic cavity132 acquired prior to or during the procedure, as described in greaterdetail below. For example, if the tip section 124 of the catheter 104 ismovable in blood in the anatomic cavity 132 and obstructed only by asurface 133 of the anatomic cavity 132, the known positions of the tipsection 124 of the catheter 104 can be taken together to provide anindication of a blood-tissue boundary of the anatomic cavity 132, andthis blood-tissue boundary can form a basis for the three-dimensionalrepresentation 134 of the anatomic cavity 132.

In general, a three-dimensional model 136 projected onto the graphicaluser interface 110 can include the three-dimensional representation 134of the anatomic cavity 132 and a representation 138 of the catheter 104.The representation 138 of the catheter 104 can include, for example, adepiction of the tip section 124, at the position of the tip section 124determined based on the signal from the magnetic position sensor 130.Examples of such a depiction of the tip section 124 can include, by wayof example and not limitation, one or more of the following: an icon; anoutline; a two-dimensional geometric shape such as a circle; and athree-dimensional geometric shape such as a sphere. Additionally, oralternatively, the representation 138 of the catheter 104 can include athree-dimensional depiction of the tip section 124. Continuing with thisexample, the three-dimensional depiction of the tip section 124 can beat least partially based on knowledge of the size and shape of the tipsection 124. Thus, for example, in implementations in which the tipsection 124 is deformed through contact with a surface of an anatomicstructure, the deformation of the tip section 124 can be shown in thethree-dimensional depiction.

It should be appreciated that the three-dimensional model 136 hasutility as, among other things, an analog for the position of the tipsection 124 of the catheter 104 in the anatomic cavity 132. That is, theposition of the tip section 124 of the catheter 104 relative to thesurface 133 of the anatomic cavity 132 is known (e.g., based on thesignal received by the interface unit 108 from the magnetic positionsensor 130) and can be represented on the graphical user interface 110as a position of the representation 138 of the catheter 104 relative tothe three-dimensional representation 134 of the anatomic cavity 132.Thus, for example, as the tip section 124 moves within the anatomiccavity 132 during a medical procedure, the representation 138 of thecatheter 104 can be depicted on the graphical user interface 110 asundergoing analogous, or at least similar, movements relative to thethree-dimensional representation 134 of the anatomic cavity 132 in thethree-dimensional model 136. Given this correspondence between thethree-dimensional model 136 and the physical aspects of the medicalprocedure, it should be appreciated that displaying multiple images ofthe three-dimensional model 136 on the graphical user interface 110 canbe a useful visualization tool for the physician as the physician movesthe tip portion 124 of the catheter 104 in the anatomic cavity 132.

In an exemplary treatment, the tip section 124 can be placed intocontact with the surface of the anatomic cavity 132 and RF energy can bedirected from the tip section 124 to the surface 133 of the anatomiccavity 132 to ablate tissue at some depth relative to the surface 133.In implementations in which the anatomic cavity 132 is a heart cavity,such ablations created by the tip section 124 along the surface 133 ofthe anatomic cavity 132 can, for example, treat cardiac arrhythmia inpatients with this condition. However, the effectiveness of theablations created using the tip section 124 along the surface 133 ofsuch a heart cavity can be dependent upon location of the ablations.Accordingly, the multi-dimensional visualization of the position of thecatheter 104 facilitated by displaying multiple images of thethree-dimensional model 136, according to any one or more of the methodsdescribed herein, can be useful for the efficient and effective mappingof the heart and/or efficient and effective delivery of ablationtreatment to treat cardiac arrhythmia.

The graphical user interface 110 can be two-dimensional such thatprojections of the three-dimensional model 136 can be displayed on thegraphical user interface 110 according to any one or more of the methodsdescribed in greater detail below. Thus, for example, the graphical userinterface 110 can be a display of a two-dimensional monitor of any ofvarious different known types. It should be appreciated, however, thatthe graphical user interface 110 can additionally or alternativelyinclude a three-dimensional display including, for example, an augmentedreality environment and/or a virtual reality environment. In general,multiple instances of all or a portion of the three-dimensional model136 can be displayed on the graphical user interface 110, with eachinstance corresponding to a view useful for providing the physician withspatial context (e.g., through reference points viewed from multiple,different perspectives) for three-dimensional maneuvering of the tipportion 124 in the anatomic cavity 132.

In instances in which the graphical user interface 110 is atwo-dimensional display, the three-dimensional model 136 can beprojected in multiple directions to form multiple, two-dimensionalimages displayed on the graphical user interface 110. For example, asshown in FIG. 5, the three-dimensional model 136 can be projected to afirst viewing window 140 of a first image plane 142 to form a firstimage 144 and to a second viewing window 146 of a second image plane 148to form a second image 150. The first viewing window 140 can correspondto a field of view of a portion of the graphical user interface 110,upon which the first image 144 is displayed. Similarly, the secondviewing window 146 can correspond to a field of view of another portionof the graphical user interface 110, upon which the second image 150 isdisplayed.

The first image plane 142 is different from the second image plane 148such that the resulting display of the first image 144 and the secondimage 150 on the two-dimensional display of the graphical user interface110 represents the three-dimensional model 136 from differentperspectives. As described in greater detail below, the second imageplane 148 can be positioned relative to the three-dimensional model 136and a clipping surface 145 such that the projection of thethree-dimensional model 136 forming the second image 150 shows a clippedportion of the three-dimensional model 136 as projected onto the secondviewing window 146. The first image 144 can show the entirety of thethree-dimensional model 136 projected onto the first viewing window 140.Thus, more generally, the first image 144 and the second image 150 canprovide visual representations of multiple, different surfaces of thethree-dimensional representation 134 of the anatomic cavity 132, withthe first image 144 showing more of the three-dimensional model (e.g.,the entirety of the three-dimensional model) than is shown in the secondimage 150. Accordingly, in combination, the first image 144 and thesecond image 150 can facilitate observation of one or more spatialreference points from multiple perspectives to provide the physicianwith spatial context useful for maneuvering the tip section 124 inthree-dimensions in the anatomic cavity 132.

In general, the clipping surface 145 can extend through thethree-dimensional model 136 to divide the model into two portions. Forexample, the clipping surface 145 can extend through thethree-dimensional representation 134 of the anatomic structure 132 and,optionally, through the representation 138 of the catheter 104. One ofthe portions of the three-dimensional model 136 can be removed relativeto the clipping surface 145 to form a clipped model 147. As a specificexample, the portion of the three-dimensional model 136 on one side ofthe clipping surface 145 can be removed to form the clipped model 147.As used herein, removing a portion of the three-dimensional model 136 toform the clipped model 147 can include any manner and form of deleting,replacing, deemphasizing (e.g., making translucent), or otherwiserendering the portion of the three-dimensional model 136 in the clippedmodel 147 as distinguished from the portion of the three-dimensionalmodel 136 that is not in the clipped model 147. The point of view of thesecond viewing window 146 can intersect the clipping surface 145 suchthat the second image 150 can include a visual representation of aregion within a boundary surface defined by the three-dimensional model136. By way of a more specific example, the clipping surface 145 can bea plane, and the second image plane 148 defining the second viewingwindow 146 can be substantially parallel to the clipping surface 145.While clipping surface 145 can be usefully formed as a plane tofacilitate visualization into the boundary surface defined by thethree-dimensional model, it should be appreciated that the clippingsurface 145 can have any of various difference shapes including, forexample, one or more curved surfaces, a series of planar surfaces, andcombinations thereof.

The clipping surface 145 can be substantially fixed relative to thesecond image plane 148. In such implementations, it should be understoodthat the clipping surface 145 can move as the second image plane 148moves. This can advantageously produce dynamic changes the clipped model147 in coordination with the second image plane 148. The result,therefore, of such dynamic changes is that the second image 150 canchange as the second image plane 148 moves, as described in furtherdetail below.

Additionally, or alternatively, the clipping surface 145 can beadjustable relative to the second image plane 148. For example, theclipping surface 145 can be adjustable based on one or more user inputsrelated to one or more of the shape and positioning of the clippingsurface relative to the second image plane 148. Such adjustability canbe useful, for example, for facilitating observation of specificportions of the three-dimensional model 136. One or more features (e.g.,point-of-view and size) of the first image 144 displayed on thegraphical user interface 110 can be a function of at least the positionof the first image plane 142 relative to the three-dimensional model 136and the size and position of the first viewing window 140 on the firstimage plane 142. Similarly, one or more features of the second image 150displayed on the graphical user interface 110 can be a function of atleast the position of the second image plane 148 relative to thethree-dimensional model 136 and the size and position of the secondviewing window 146 on the second image plane 148. As the tip section 124is moved within the anatomic cavity 132, the position of one or both ofthe first image plane 142 and the second image plane 148 can changerelative to the three-dimensional model 136 and, additionally, oralternatively, the size and position of one or both of the first viewingwindow 140 and the second viewing window 146 on the respective one ofthe first image plane 142 and the second image plane 148 can change. Theresult of such changes can include corresponding changes to thepoint-of-view and/or size of a respective one of the first image 144 andthe second image 150 displayed on the graphical user interface 110.

At any of various different time steps, the orientation of the firstimage plane 142 can be adjusted relative to the three-dimensional model136. As an example, adjusting the first image plane 142 can be based onthe location of the catheter 104 in the anatomic cavity 132 of thepatient 102. Continuing with this example, the orientation of the secondimage plane 148 can be determined based on the orientation of the firstimage plane 142 such that both the first image plane 142 and the secondimage plane 148 are adjusted relative to the three-dimensional model 136based on the location of the catheter 104 in the anatomic cavity 132 ofthe patient 102. It should be appreciated that adjustment of one or bothof the first image plane 142 and the second image plane 148 based on thelocation of the catheter 104 in the anatomic cavity 132 can be usefulfor providing dynamically changing views of the three-dimensional model136 shown in the first image 144 and the second image 150 displayed onthe graphical user interface 110. Any of various different aspects offorming the first image 144 and/or the second image 150 displayed on thegraphical user interface 110 can be filtered such that the dynamicallychanging views of the three-dimensional model 136 shown in therespective first image 144 and the second image 150 can be shown asundergoing smooth motion. For example, smooth transitions of one or bothof the first image 144 and the second image 150 can be achieved byfiltering one or more of the following: the location of the catheter 104in the anatomic cavity 132, adjustment of one or both of the first imageplane 142 and the second image plane 148, and adjustment of one or bothof the first viewing window 140 and the second viewing window 146.

In certain implementations, the second image plane 148 can be in a fixedorientation (e.g., at a fixed angle) relative to the first image plane142. In such implementations, the position of the first image plane 142can be adjusted relative to the three-dimensional model 136 while thesecond image plane 148 remains in a fixed orientation relative to thefirst image plane 142, thus producing a fixed change in the second image150 when the first image 144 changes on the graphical user interface110. Such fixed orientation of the second image plane 148 relative tothe first image plane 142 can be useful, in certain instances, forproviding a fixed reference for the second image 150. With such a fixedreference for the second image 150, the physician can combine theinformation presented in the first image 144 and the second image 150 bytracking only a single coordinate system.

While the second image plane 148 can be in a fixed orientation relativeto the first image plane 142, other implementations are additionally oralternatively possible. For example, an included angle between the firstimage plane 142 and the second image plane 148 can be variable (e.g.,between a range of angles). As a more specific example, the includedangle between the first image plane 142 and the second image plane 148can be an input provided by the physician according, for example, to avisualization preference of the physician.

Whether in a fixed or variable orientation relative to one another, anincluded angle between the second image plane 148 and the first imageplane 142 can be less than or equal to 90 degrees and greater than about60 degrees. Thus, for example, in implementations in which the secondimage plane 148 is in a fixed orientation relative to the first imageplane 142, the second image plane 148 can be substantially orthogonal tothe first image plane 142. Such a substantially orthogonal orientationof the second image plane 148 relative to the first image plane 142 canbe useful, for example, for providing the physician with a fixedcoordinate system with which to compare the first image and the secondimage and, thus, appreciate spatial context provided by the combinationof the first image and the second image.

In some implementations, the second image plane 148 can be restricted toa specific orientation relative to the three-dimensional model 136. Asan example, the second image plane can be restricted to a directionsuperior to the representation of the catheter 104 included in thethree-dimensional model 136 such that the second image 150 correspondingto the second image plane 148 is in a fixed orientation relative to thecatheter 104. Such fixed orientation of the second image plane 148relative to the representation of the catheter 104 can facilitate, incertain instances, locating a specific portion (e.g., an ablationelectrode) of the catheter 104 relative to one or more surfaces of theanatomic cavity 132 represented in the three-dimensional model 136.

While a single clipping surface is depicted in FIG. 5 for the sake ofclarity of illustration and explanation, it should be appreciated thatmultiple clipping surfaces can be used to remove portions ofthree-dimensional model as necessary to facilitate visualization of oneor more aspects of the three-dimensional model. For example, as shown inFIG. 6, a first clipping surface 145 a and a second clipping surface 145b each intersect the three-dimensional model 136. A portion of thethree-dimensional model 136 relative to the first clipping surface 145 acan be removed. Similarly, a portion of the three-dimensional model 136relative to the second clipping surface 145 b can be removed. With theportions of the three-dimensional model 136 removed, the clipped modelis a slice 149. In general, the slice 149 can be projected onto aviewing window (e.g., the second viewing window 146 in FIG. 5) accordingto any one or more of the various methods described herein. Theprojection of the slice 149 onto a viewing window can be useful, forexample, for representing complex anatomic geometry and, thus, forproviding a physician with useful views of an anatomic structure (e.g.,the anatomic structure 132 in FIG. 3).

In general, unless otherwise indicated, each of the following exemplarymethods can be implemented using the system 100 (FIG. 1) and/or one ormore components thereof. Thus, it should be understood that thethree-dimensional model 136 can be stored in a memory such as thestorage medium 111 (FIG. 1). It should be further or alternativelyunderstood that projection of the three-dimensional model 136 to formthe first image 144 and the second image 150 on the graphical userinterface 110, according to any one or more of the methods describedherein, can be carried out by the processing unit 109 (FIG. 1) executingcomputer-executable instructions stored on the storage medium 111 (FIG.1). The instructions stored on the storage medium 111 and executable bythe processing unit 109 to display the first image 144 and the secondimage 150 of the three-dimensional model 136 can be, for example, anapplication built using Visualization Toolkit, an open-source 3Dcomputer graphics toolkit, available at www.vtk.org. FIG. 7 is aflowchart of an exemplary method 160 of displaying multiple images of athree-dimensional model on a graphical user interface. In addition to oras an alternative to facilitating visualization in a heart chamber, theexemplary method 160 can be carried out to facilitate visualization ofany anatomic structure of a patient such as, for example, the brain, thelungs, the sinuses, and/or other hollow anatomic structures of thepatient through which a medical device, such as a catheter or othersimilar device, may be passed. More specifically, the exemplary method160 can facilitate visualization of the position of a medical devicewithin the anatomic structure from multiple perspectives, including aperspective outside of an anatomic structure. Such visualization canprovide unique advantages, for example, in maneuvering a medical deviceduring any of various different medical procedures guided by, orotherwise performed in conjunction with, a three-dimensional model(e.g., a three-dimensional model representing a blood-tissue boundary ina heart cavity). The medical device, it should be appreciated, can beany medical device that is typically inserted into an anatomic structurefor the purpose of diagnosis, treatment, or both and, thus, can includethe catheter 104 described above with respect to FIG. 2.

The exemplary method 160 can include receiving 162 a signal indicativeof location of a medical device in an anatomic structure of a patient,constructing 164 a three-dimensional model, clipping 166 thethree-dimensional model to form a clipped model, and displaying 168, ona graphical user interface, a first image including thethree-dimensional model and a second image including the clipped model.Clipping the three-dimensional model can include removing a firstportion of the three-dimensional model relative to a first clippingsurface intersecting the three-dimensional model to form the clippedmodel. As described in greater detail below, one or both of constructing164 the three-dimensional model and clipping 166 the portion of thethree-dimensional model to be displayed in the second image can be basedon the received 162 signal indicative of the location of the medicaldevice in the anatomic structure of the patient such that, for example,the first image and the second image displayed on the graphical userinterface can provide updated (e.g., substantially real-time) spatialcontext to the physician as the medical device is moved in the anatomicstructure. As used herein, the three-dimensional model shall beunderstood to include a three-dimensional representation of the anatomicstructure and, optionally, a representation of the medical devicerelative to the anatomic structure of the patient.

In general, the first image plane and the second image plane associatedwith the exemplary method 160 can be any one or more of the image planesdescribed herein (e.g., the first image plane 142 and the second imageplane 148 shown in FIG. 5) and can have any manner or form oforientations relative to one another as described herein to producecorresponding changes to the respective first image and second image.Thus, for example, the second image plane can intersect the first imageplane such that the corresponding first image and the correspondingsecond image represent different perspectives of the three-dimensionalmodel to provide spatial context useful for maneuvering the medicaldevice in multiple directions within the anatomic structure.

Receiving 162 the signal indicative of the location of the medicaldevice in the anatomic structure can include receiving a signalindicative of the location of the medical device according to any one ormore of the methods described herein. For example, the location of themedical device can be the location of a tip section of a catheter (e.g.,the tip section 124 of the catheter 104 of FIG. 2). It should beunderstood, however, that the location of the medical device can also orinstead include the location of any predetermined portion of the medicaldevice in the anatomic structure.

Receiving 162 the signal indicative of the location of the medicaldevice in the anatomic structure can include receiving the signal over aperiod of time. In particular, the signal indicative of the location ofthe medical device can be a time-varying signal. For example, thereceived 162 signal can be time-varying and processed such that thefirst image, the second image, or both can be generated or updated basedon the processed received 162 signal. In general, any one or more ofvarious, different functions can be applied to the time-series of thereceived 162 signal indicative of the location of the medical device toavoid, or at least lessen, the impact of abrupt motion that may beassociated with changes in location of the medical device. For example,one or both of the first image and the second image can be based onlow-pass filtering the received 162 signal indicative of the location ofthe medical device. Additionally, or alternatively, functions applied tothe time-series of the received 162 signal indicative of the location ofthe medical device can include one or more of infinite impulse response(IIR) filters, finite impulse response (FIR) filters, non-linear filters(e.g., a median filter), and combinations thereof. More generally still,while the received 162 signal indicative of the location of the medicaldevice is described as being filtered to control updates of the firstimage, the second image, or both, it should be appreciated that anyother aspect of forming the first image and the second image, as thecase may be, may be additionally, or alternatively, filtered to achievesmooth transitions of the respective first image and the second image.For example, the position of an image plane (e.g., one or both of theimage planes 142 and 148 in FIG. 5) and/or the size of a viewing window(e.g., one or both of the viewing windows 140 and 146 in FIG. 5) can befiltered such that the respective first image and second image displayedon the graphical user interface can be depicted as moving smoothly, evenwhen the underlying movement of the catheter is characterized by anumber of rapid, small movements and/or large, abrupt transitions.

Processing the time-varying signal can be useful, for example, forimproved perception of the first image, the second image, or both on thegraphical user interface. For example, processing the time-varyingsignal of the received location of the medical device can smooth outchanges in the first image and/or the second image corresponding tochanges in location of the medical device such that the resultingdisplay of the first image and the second image on the graphical userinterface can be more stable (e.g., less shaky), as compared to imagesdisplayed on the graphical user interface based on an unprocessed,time-varying signal. Accordingly, processing the time-varying signal canbe useful for creating changes to one or more of the first image and thesecond image at a rate that is both rapid enough to keep pace withchanges in position of the medical device but slow enough to avoid largechanges and/or many small, rapid changes to the first image and/or thesecond image that are more likely to interfere with the physician's useof the three-dimensional model to position the medical device.

In implementations in which the second image plane is fixed relative tothe first image plane, it should be appreciated that the second imagecan be stabilized by, for example, processing the received 162 signalindicative of location of the medical device and, additionally oralternatively, by processing the first image itself. More generally,processing associated with the first image can stabilize the secondimage.

In some implementations, the received 162 signal can be processeddifferently with respect to the first image and the second image. Forexample, processing associated with the first image can result in thefirst image being relatively responsive to changes in the received 162signal indicative of the location of the medical device while processingassociated with the second image can result in the second image being,relative to the first image, less responsive to such changes in thereceived 162 signal. Alternatively, the processing associated with therespective images can be such that the second image is relatively moreresponsive to changes in the received 162 signal while the first imageis relatively less responsive to the received 162 signal.

In general, the first image and the second image on the graphical userinterface can be updated based on a change in the received 162 locationsignal corresponding to the location of the medical device. The updatecan occur at each time step or, in implementations in which the receivedsignal is processed, the update can occur based on a combination of timesteps. In each update of the first image and the second image, one ormore of the steps of the exemplary method 160 can be repeated.Additionally, or alternatively, updating the first image and the secondimage can be based on a user input.

Constructing 164 the three-dimensional model can include updating therepresentation of the medical device based on the received 162 signalindicative of the location of the medical device. Thus, for example, asthe medical device can be moved in the anatomic structure, such movementof the medical device can be translated into movement of therepresentation of the medical device relative to the three-dimensionalrepresentation of the anatomic structure. The movement of therepresentation of the medical device relative to the three-dimensionalrepresentation of the anatomic structure can be represented on thegraphical user interface as updates are made to one or both of the firstimage and the second image. Accordingly, the movement of therepresentation of the medical device shown in the first image and thesecond image on the graphical user interface can serve as an analog forthe physical movement of the medical device in the anatomic structure.

Constructing 164 the three-dimensional model can be based on thereceived 162 signal as processed according to any one or more of themethods described herein. Accordingly, as an example, constructing 164all or a portion of the three-dimensional model can be based on low-passfiltering the received 162 signal. In certain instances, constructing164 the three-dimensional model based on the received 162 signal asprocessed according to any one or more of the methods described hereincan be useful for representing smooth movements of the representation ofthe medical device through successive updates of the first image and thesecond image shown on the graphical user interface.

In certain implementations, constructing 164 the three-dimensional modelcan include receiving one or more images (e.g., computed tomography (CT)images, magnetic resonance imaging (MRI) images, and/or boundarysurfaces derived therefrom) of the anatomic structure and registeringthe images to a coordinate system of a sensor providing a signalindicative of location of the medical device (e.g., the magneticposition sensor 130 of FIG. 2). Thus, in such implementations, thethree-dimensional representation of the anatomic structure can be basedon the one or more images and constructing 164 the three-dimensionalmodel can include rendering the representation of the medical devicesuperimposed on the one or more images. The one or more received imagescan be acquired, for example, prior to the procedure. It should beappreciated, however, that these images can be acquired in real-time(e.g., using rotational angiography). Additionally, or alternatively,the three-dimensional representation can include one or more boundarysurfaces generated according to any one or more of the methods describedherein.

In some implementations, constructing 164 the three-dimensional modelcan include adding visual indicia (e.g., a tag) to the three-dimensionalrepresentation of the anatomic structure and, or instead, to anotherportion of the three-dimensional model. In certain implementations, theexemplary method 160 can include receiving a signal indicative of alocation of a treatment applied by the medical device to the anatomicstructure. In such implementations, constructing 164 thethree-dimensional model can include adding visual indicia to thethree-dimensional model of the anatomic structure in a locationcorresponding to the location of the treatment in the anatomicstructure. The visual indicia can be shown on the projection of thethree-dimensional representation of the anatomic structure shown in oneor both of the first image and the second image. Thus, the position ofthe visual indicia on the three-dimensional representation of theanatomic structure can be observed from multiple perspectives. Suchmultiple perspectives can, it should be appreciated, facilitateapplication of one or more subsequent treatments relative to the visualindicia. For example, the multiple perspectives observable in the firstimage and the second image can facilitate application of an ablationpattern (e.g., a pattern of overlapping lesions), such as during aprocedure to treat cardiac arrhythmia.

Constructing 164 the three-dimensional model can, additionally oralternatively, include adding visual indicia manually to thethree-dimensional representation. For example, a physician or techniciancan add visual indicia corresponding to the location of an anatomicfeature to the three-dimensional model (e.g., to the three-dimensionalrepresentation of the anatomic structure). For example, the exemplarymethod 160 can further include receiving a signal indicative of locationto be tagged such that the constructed 164 three-dimensional model caninclude the visual indicia added to the three-dimensional model in alocation on the three-dimensional model corresponding to the locationbeing tagged. Additionally, or alternatively, the visual indicia can beadded to the three-dimensional representation of the anatomic structurein one or both of the first image and the second image. Thus, forexample, the physician or technician can add visual indicia to thethree-dimensional representation in the second image if the viewpresented in the second image is more convenient for tagging than theview presented in the first image.

The visual indicia corresponding to the anatomic feature can bedisplayed in the respective projections of the three-dimensional modelforming the first image and the second image. Thus, for example, thefirst image and the second image displayed on the graphical userinterface can provide the physician with multiple perspectives of themedical device relative to the anatomic feature represented by thevisual indicia. Such multiple perspectives can be useful, in certaininstances, when moving the medical device toward or away from theanatomic feature.

In general, clipping 166 the three-dimensional model to form a clippedmodel to be displayed can address, among other things, a challengeassociated with certain medical procedures (e.g., cardiac procedures) inwhich it is desirable to make observations from a perspective lookinginto a surface within which a medical device is contained. To addressthis challenge, clipping 166 the three-dimensional model can includeidentifying a portion (e.g., a continuous portion) of thethree-dimensional model to remove. Removal, in this context, should beunderstood to include deleting, deemphasizing (e.g., makingtranslucent), or otherwise modifying in the second image todifferentiate the second image from the first image. Suchdifferentiation may be advantageous when, for example, a portion of areceived or acquired image is displayed in the second image. Continuingwith this example, removing a portion of a received or acquired image inthe second image may facilitate visualization of volumetric informationsuch as tissue thickness or ischemia, which may otherwise be difficultto visualize on a boundary surface. Additionally, or alternatively,differentiation of the second image from the first image can improve thephysician's ability to observe the position of the representation of themedical device relative to the three-dimensional model of anatomicstructure. With this improved ability to make observations based on thethree-dimensional model, it should be appreciated that the physician'sknowledge of the position of the medical device relative to the anatomicstructure during a medical procedure can be accordingly improved. Forexample, when the physician wishes to verify or modify the position ofthe medical device relative to the anatomic structure in a directionperpendicular to the first image plane, the differentiation of thesecond image can provide a clearer visual indication of this relativeposition than may be available in the first image.

The three-dimensional representation of the anatomic structure caninclude a boundary surface which can represent a contour of the anatomicstructure being modeled. As an example, the boundary surface canrepresent the blood-tissue boundary such as the one depicted in thethree-dimensional representation 134 of the anatomic cavity 132described above with respect to FIGS. 3-5. Generally, clipping 166 theportion of the three-dimensional model to be displayed in the secondimage can include selecting less than the entirety of the boundarysurface for representation in the second image. That is, the first imagecan include a portion of the boundary surface that is not shown in thesecond image.

Clipping 166 the three-dimensional model can include selecting one ormore of a received or acquired image and a boundary surface generatedaccording to any one or more of the methods described herein. Morespecifically, clipping 166 the three-dimensional model can includeremoving a first portion of the three-dimensional model relative to afirst clipping surface such that the portion of the three-dimensionalmodel that is not removed forms a clipped model.

The position of one or more of the second image plane and the firstclipping surface can be based on the received 162 signal indicative ofthe location of the medical device.

In implementations in which the second image plane and the firstclipping surface are fixed relative to one another, intersection of thefirst clipping surface with the three-dimensional model can beorthogonal to the first image plane. In such instances, the second imagecan include a projection of a portion of the boundary surface of thethree-dimensional model extending in a direction away from andorthogonal to the second image plane. This arrangement of the secondimage plane relative to the first image plane can advantageously provideviews of the three-dimensional model and the corresponding clipped modelin a single coordinate system. Taking the first image 144 and the secondimage 150 shown in FIGS. 4 and 5 as an example, the physician canobserve an outer portion of the three-dimensional representation of theanatomic structure in the first image including the three-dimensionalmodel while observing, from a readily understood second perspective, theposition of the medical device relative to an inner portion of thethree-dimensional representation of the anatomic structure in the secondimage including the clipped model.

In some implementations, the first clipping surface can extend throughthe representation of the medical device. Such a first clipping surfaceextending through the representation of the medical device can ensurethat the position of the medical device is clearly observable in thesecond image. That is, because the first clipping surface extendsthrough the representation of the medical device, the position of themedical device in the second image can be generally unobscured byportions of the three-dimensional model corresponding to thethree-dimensional representation of the anatomic structure. In certainimplementations in which the first clipping surface extends through therepresentation of the medical device, the medical device can be renderedin any of various different forms. For example, the medical device canbe rendered in the second image as clipped. Additionally, oralternatively, the medical device can be rendered in the second image asunclipped.

Further, or instead, the first clipping surface can be (e.g., when thefirst clipping surface is fixed relative to the second image plane)based on the received 162 signal indicative of the location of themedical device. Accordingly, the first clipping surface can be selectedbased the received 162 location signal that has been processed accordingto any one or more of the methods described herein, resulting in any oneor more of the stabilization advantages described herein with respect toprocessing the received 162 location signal. Additionally, oralternatively, the position of the first clipping surface relative tothe three-dimensional model can change based on changes in the locationof the medical device. For example, the first clipping surface can movein accordance with movement of the medical device.

Displaying 168 the first image and the second image can includedisplaying images on any of various different graphical user interfacesdescribed herein. In general, displaying 168 the first image and thesecond image can include displaying different parts of thethree-dimensional model in the first image and the second image. If aportion of the three-dimensional model is displayed in the first image,it should be understood that the clipped model includes a subset of aportion of the three-dimensional model displayed in the first image.While a subset of the three-dimensional model is displayed in the secondimage, it should be appreciated that the second image can include otherinformation useful for guiding a medical procedure. For example, thesecond image can, optionally, include a display of a received oracquired image that is not shown in the first image. Together, aboundary surface of the clipped model and the received or acquired imagecan be useful, for example, for providing a physician with feedbackregarding the position of the medical device and local conditions of theanatomic structure in the vicinity of the medical device.

The first image and the second image can be displayed 168 on atwo-dimensional display such as the graphical user interface 110described herein with respect to FIGS. 1, 4, and 5. Further, the firstimage can be the projection of the three-dimensional model on the firstviewing window of the first image plane according to any one or more ofthe methods described herein. Similarly, the second image can be theprojection of the clipped model on the second viewing window of thesecond image plane according to any one or more of the methods describedherein.

In some implementations, displaying 168 the first image and the secondimage on the graphical user interface can include displaying the firstimage and the second image simultaneously on the graphical userinterface. With such a simultaneous display, the physician canadvantageously compare the first image to the second image to facilitatemulti-directional movement of the medical device in the anatomicstructure. Additionally, or alternatively, the simultaneous display ofthe first image and the second image can reduce the need for aphysician, or a technician assisting the physician, to manipulate thegraphical user interface to switch between views.

Displaying 168 the first image and the second image can includedisplaying the first image and the second image as different sizes onthe graphical user interface. For example, given that the first imageincludes a projection of the three-dimensional model and the secondimage includes a projection of only the clipped model, the first imagecan be displayed as larger than the second image. Such sizing of thefirst image relative to the second image can be useful for displayinganatomic features in a sufficient size and with sufficient detail to beuseful to the physician while allowing the physician to use the secondimage as an auxiliary view useful for navigating the medical device inthe anatomic structure.

Displaying 168 the first image and the second image can includeadjusting a zoom magnification in the respective image. For example,displaying 168 the first image and the second image can includeadjusting a distance from at least one of the first image plane or thesecond image plane to the three-dimensional model. Also, oralternatively, displaying 168 the first image and the second image caninclude sizing at least one of the first viewing window or the secondviewing window to change a field of view depicted in the respectiveimage. Further, or instead, displaying 168 the first image and thesecond image can include moving a center of projection relative to arespective viewing window.

In certain implementations, sizing the first viewing window and, thus,the first image can be based on at least one dimension of thethree-dimensional model in the first image plane. For example, the sizeof the first viewing window can be a multiple of a largest dimension ofthe three-dimensional model in the first image plane such that theentire three-dimensional model is projected onto the first viewingwindow to form the first image. In such implementations, it should beappreciated that the size of the first image can vary as thethree-dimensional model is moved (e.g., rotated) relative to the firstimage plane. Additionally, or alternatively, the size of the firstviewing window can be based on a user input such that, in response tothe user input, the size of the three-dimensional model when projectedonto the first viewing window to form the first image is varied.

In some implementations, sizing the second viewing window can be basedon a bounding volume defined around the three-dimensional model. Thebounding volume can be a sphere, a box, or any predetermined geometricvolume. The size of the second viewing window can be based on adimension of the bounding volume. For example, the size of the secondviewing window can be based on a maximum dimension of the boundingvolume. In instances in which the bounding volume is a sphere, the sizeof the second viewing window can be based on a diameter (e.g., a fixedmultiple of the diameter) of the sphere bounding the three-dimensionalmodel. In general, basing the size of the second viewing window on themaximum dimension of the bounding volume can facilitate maintaining afixed size of the second image as the three-dimensional model isrotated.

The second image can include visual indicia highlighting a boundarybetween the representation of the medical device and thethree-dimensional representation of the anatomic structure. Such aboundary, it should be understood, can be useful for providing aphysician with a visual delineation between the representation of themedical device and the three-dimensional representation of the anatomicstructure when the medical device is in close proximity to the surfaceof the anatomic structure. Such visual delineation can be particularlyuseful in instances in which the second image is smaller than the firstimage, thus making details of the second image more difficult toperceive than similar details represented in the first image.Additionally, or alternatively, such visual indicia can provide a clearindication of the spacing and relative orientation of the medicaldevice, which can be useful, for example, for facilitating fine movementof the medical device relative to the surface of the anatomic structure(e.g., for positioning the medical device into contact with the anatomicstructure).

In certain implementations, the second image can include visual indiciahighlighting a contour of the boundary surface within a predetermineddistance of the first clipping surface (e.g., at the intersection of theboundary surface and the first clipping surface). Such a highlightedcontour can be a useful visualization guide for the physician. Forexample, a highlighted contour of this type can be useful for providingthe physician with an indication of the shape of the anatomic structurealong the first clipping surface.

In some implementations, the second image can include visual indiciahighlighting an outer region of the medical device and the surface ofthe anatomic structure. In such implementations, as the visual indiciahighlighting the boundary of the medical device moves near the surfaceof the anatomic structure in accordance with corresponding movement ofthe medical device, the boundary between these two surfaces can bereadily perceivable in the second image. It should be appreciated thatcertain combinations of visual indicia can advantageously differentiatethe outer surface of the medical device from the surface of the anatomicstructure. As an example, the visual indicia delineating contours of themedical device can include thick lines rendered in a color contrasting acolor palette used to represent the surface of the anatomic structure inthe second image. Additionally, or alternatively, the visual indiciahighlighting the boundary of the medical device can vary in color acrosstime and/or around the boundary of the medical device to indicate avalue that varies with time and/or location on the medical device,respectively. Such indicated values can be derived, for example, frommeasurements collected with the medical device during a procedure suchas, for example: contact force; impedance; biological electricalactivity; and/or acoustic or optical imaging data.

Clipping 166 the three-dimensional model can, in some instances, furtherinclude removing a second portion of the three-dimensional modelrelative to a second clipping surface. In such instances, the clippedmodel can be substantially between the first clipping surface and thesecond clipping surface. That is, the clipped model can be in the formof a slice, useful for visualizing complex anatomic geometry. As usedherein, “substantially between” includes variations of about ±1 mm withrespect to each of the first clipping surface and the second clippingsurface.

FIG. 8 is a flowchart of another exemplary method 170 of displayingmultiple images of a three-dimensional model on a graphical userinterface. In general, unless otherwise specified or made clear from thecontext, the exemplary method 170 can facilitate visualization of amedical device in an anatomic structure in a manner analogous to thefacilitated visualization achieved by the exemplary method 160. Further,like the exemplary method 160 (FIG. 7), the exemplary method 170 can beimplemented using the system 100 (FIG. 1) and any one or more componentsthereof. Further still, to the extent the exemplary method 170 differsfrom the exemplary method 160 (FIG. 7), it will be understood that anyone or more steps of the exemplary method 170 may be combined with orreplace any one or more steps of the exemplary method 160 to displaymultiple images of a three-dimensional model on a graphical userinterface.

The exemplary method 170 can include receiving 172 a signal indicativeof a location of a medical device in an anatomic structure of a patient,updating 174 a three-dimensional model, forming 176 a a first imageincluding a projection of the updated three-dimensional model, forming176 b a second image including a projection of the updatedthree-dimensional model, and displaying 178 the first image and thesecond image on a graphical user interface.

Receiving 172 the signal indicative of the location of the medicaldevice in the anatomic structure can include any of the variousdifferent methods of receiving a location signal described herein. Suchan exemplary method of receiving 172 the signal indicative of locationcan include, therefore, a signal received from a magnetic positionsensor such as the magnetic position sensor 130 described with respectto FIG. 2.

The three-dimensional model can be updated 174 based on the received 172signal indicative of the location of the medical device in the anatomicstructure and can include a three-dimensional representation of theanatomic structure and a representation of the medical device relativeto the anatomic structure. The three-dimensional model can be any of thevarious different three-dimensional models described herein such that,for example, the three-dimensional model is inclusive of thethree-dimensional model 136 described with respect to FIGS. 4-6.Accordingly, the three-dimensional representation of the anatomicstructure can be based on one or more received images (e.g., computedtomography (CT) images and/or magnetic resonance imaging (MRI) images)of the anatomic structure, with the images registered to a coordinatesystem of a sensor providing a signal indicative of location of themedical device (e.g., the magnetic position sensor 130 of FIG. 2). Thus,in such instances, updating 174 the three-dimensional model can includeupdating the representation of the medical device superimposed on theone or more images. The one or more received images can be acquired, forexample, prior to the procedure. It should be appreciated, however, thatthese images can be acquired in real-time (e.g., using rotationalangiography).

Updating 174 the three-dimensional model can be based on the received172 signal indicative of the location of the medical device in theanatomic structure. Accordingly, in certain implementations, updating174 the three-dimensional model can include processing the received 172signal according to one or more of the methods of processing describedherein such as, for example, low-pass filtering. That is, the updated174 three-dimensional model can be based on the received 172 signal thathas been processed. Additionally, or alternatively, processing can beapplied to one or more steps of the exemplary method 170 to achievecorresponding stabilization of the first image, the second image, orboth on the graphical user interface.

Forming 176 a the first image and forming 176 b the second image caninclude projecting the three-dimensional model according any one or moreof the methods described herein and, in particular, such methodsdescribed with respect to FIG. 5. The first image can include aprojection, on a first viewing window of a first image plane, of atleast one portion of the updated three-dimensional model. The secondimage can include another portion of the updated three-dimensionalmodel. In certain implementations, the portion of the three-dimensionalmodel projected on the second image plane can be less than the at leastone portion of the updated three-dimensional model projected on thefirst viewing window such that, for example, an inner portion of thethree-dimensional model can be more readily observable in the secondimage than in the first image.

The first image plane and the second image plane can be any one or moreof the image planes described herein such that, as one example, thefirst image plane and the second image plane can intersect one anotherto produce complementary views of the three-dimensional model in thefirst image and the second image. Further, the first image plane can beadjusted according to any of the various different methods describedherein. Such adjustment of the first image plane, it should beappreciated, can produce a corresponding change in the second imageplane. For example, the first image plane can be adjusted while thesecond image plane is maintained in a fixed orientation (e.g., a fixedorthogonal relationship) relative to the first image plane, and thecorresponding second image can change in a fixed relationship to thefirst image as the first image plane changes.

The first image plane can be orientated relative to the received 172location of the medical device. In such instances, the first image canprovide a view of the three-dimensional model from a perspective of themedical device as the medical device is moved relative to the anatomicstructure. Such a perspective can be useful, for example, for finethree-dimensional manipulation of the medical device relative to theanatomic structure (e.g., such as to establish contact between themedical device and the anatomic structure during an ablation treatment).In certain implementations, orienting the first image plane relative tothe received 172 location of the medical device can include orientingthe first image plane in a direction perpendicular to an average ofsurface normal vectors along a portion, closest to the medical device,of a boundary surface of the anatomic structure in the three-dimensionalmodel. That is, more generally, orienting the first image plane relativeto the received 172 location of the medical device can include movingthe first image plane based on one or more local features of theanatomic structure in the vicinity of the medical device, facilitating,for example, three-dimensional manipulation of the medical devicerelative to a surface of the anatomic structure.

Displaying 178 the first image and the second image on the graphicaluser interface can be carried out according to any one or more of themethods described herein. Thus, for example, displaying 178 the firstimage and the second image can include displaying the first image andthe second image simultaneously on the graphical user interface.Additionally, or alternatively, the graphical user interface can be anyone or more of the graphical user interfaces described herein including,for example, the graphical user interface 110 described with respect toFIGS. 1 and 5.

The above systems, devices, methods, processes, and the like may berealized in hardware, software, or any combination of these suitable fora particular application. The hardware may include a general-purposecomputer and/or dedicated computing device. This includes realization inone or more microprocessors, microcontrollers, embeddedmicrocontrollers, programmable digital signal processors or otherprogrammable devices or processing circuitry, along with internal and/orexternal memory. This may also, or instead, include one or moreapplication specific integrated circuits, programmable gate arrays,programmable array logic components, or any other device or devices thatmay be configured to process electronic signals.

It will further be appreciated that a realization of the processes ordevices described above may include computer-executable code createdusing a structured programming language such as C, an object orientedprogramming language such as C++, or any other high-level or low levelprogramming language (including assembly languages, hardware descriptionlanguages, and database programming languages and technologies) that maybe stored, compiled or interpreted to run on one of the above devices,as well as heterogeneous combinations of processors, processorarchitectures, or combinations of different hardware and software. Inanother aspect, the methods may be embodied in systems that perform thesteps thereof, and may be distributed across devices in a number ofways. At the same time, processing may be distributed across devicessuch as the various systems described above, or all of the functionalitymay be integrated into a dedicated, standalone device or other hardware.In another aspect, means for performing the steps associated with theprocesses described above may include any of the hardware and/orsoftware described above. All such permutations and combinations areintended to fall within the scope of the present disclosure.

Embodiments disclosed herein may include computer program productscomprising computer-executable code or computer-usable code that, whenexecuting on one or more computing devices, performs any and/or all ofthe steps thereof. The code may be stored in a non-transitory fashion ina computer memory, which may be a memory from which the program executes(such as random access memory associated with a processor), or a storagedevice such as a disk drive, flash memory or any other optical,electromagnetic, magnetic, infrared or other device or combination ofdevices.

In another aspect, any of the systems and methods described above may beembodied in any suitable transmission or propagation medium carryingcomputer-executable code and/or any inputs or outputs from same.

The method steps of the implementations described herein are intended toinclude any suitable method of causing such method steps to beperformed, consistent with the patentability of the following claims,unless a different meaning is expressly provided or otherwise clear fromthe context. So for example performing the step of X includes anysuitable method for causing another party such as a remote user, aremote processing resource (e.g., a server or cloud computer) or amachine to perform the step of X. Similarly, performing steps X, Y and Zmay include any method of directing or controlling any combination ofsuch other individuals or resources to perform steps X, Y and Z toobtain the benefit of such steps. Thus method steps of theimplementations described herein are intended to include any suitablemethod of causing one or more other parties or entities to perform thesteps, consistent with the patentability of the following claims, unlessa different meaning is expressly provided or otherwise clear from thecontext. Such parties or entities need not be under the direction orcontrol of any other party or entity, and need not be located within aparticular jurisdiction.

It should further be appreciated that the methods above are provided byway of example. Absent an explicit indication to the contrary, thedisclosed steps may be modified, supplemented, omitted, and/orre-ordered without departing from the scope of this disclosure.

It will be appreciated that the methods and systems described above areset forth by way of example and not of limitation. Numerous variations,additions, omissions, and other modifications will be apparent to one ofordinary skill in the art. In addition, the order or presentation ofmethod steps in the description and drawings above is not intended torequire this order of performing the recited steps unless a particularorder is expressly required or otherwise clear from the context. Thus,while particular embodiments have been shown and described, it will beapparent to those skilled in the art that various changes andmodifications in form and details may be made therein without departingfrom the spirit and scope of this disclosure and are intended to form apart of the invention as defined by the following claims.

What is claimed is:
 1. A method comprising: receiving a signalindicative of a location of a medical device in an anatomic structure ofa patient; based at least in part on the received location signal,constructing a three-dimensional model including a three-dimensionalrepresentation of the anatomic structure and a representation of themedical device relative to the anatomic structure; based at least inpart on the received location signal, clipping the three-dimensionalmodel, clipping the three-dimensional model including removing a firstportion of the three-dimensional model relative to a first clippingsurface intersecting the three-dimensional model to form a clippedmodel; and displaying, on a graphical user interface, a first image anda second image, wherein the first image includes a projection of thethree-dimensional model on a first viewing window of a first imageplane, and the second image includes a projection of the clipped modelon a second viewing window of a second image plane.
 2. The method ofclaim 1, wherein the three-dimensional representation of the anatomicstructure includes a boundary surface and displaying the first imageincludes displaying a portion of the boundary surface that is not shownin the second image.
 3. The method of claim 1, wherein constructing thethree-dimensional model includes receiving one or more images of theanatomic structure and registering the images to a coordinate system ofa sensor providing the signal indicative of the location of the medicaldevice.
 4. The method of claim 1, wherein constructing thethree-dimensional model includes updating the representation of themedical device based on the received location signal.
 5. The method ofclaim 1, wherein the received location signal is a time-varying signaland clipping the three-dimensional model further includes processing thetime-varying received location signal and selecting the first clippingsurface based on the processed, received location signal.
 6. The methodof claim 5, wherein processing the time-varying received location signalincludes low-pass filtering the received location signal.
 7. The methodof claim 1, wherein the three-dimensional model includes a boundarysurface of the anatomic structure and displaying the second imageincludes highlighting a contour of the boundary surface within apredetermined distance of the first clipping surface.
 8. The method ofclaim 7, wherein highlighting the contour of the boundary surface withinthe predetermined distance of the first clipping surface includeshighlighting a contour of the boundary surface intersected by the firstclipping surface.
 9. The method of claim 1, further comprising adjustingthe first image plane and maintaining the second image plane in a fixedorientation relative to the first image plane.
 10. The method of claim9, wherein the second image plane is restricted to a direction superiorto the representation of the location of the medical device in thethree-dimensional model.
 11. The method of claim 9, wherein adjustingthe first image plane includes orienting the first image plane relativeto the location, in the anatomic structure of the patient, of themedical device.
 12. The method of claim 1, wherein displaying the firstimage and the second image on the graphical user interface includesadjusting the size of the three-dimensional model as projected onto oneor both of the first viewing window and the second viewing window. 13.The method of claim 1, further comprising receiving a signal indicativeof a location of a treatment applied by the medical device to theanatomic structure, wherein constructing the three-dimensional modelincludes adding visual indicia to the three-dimensional model, thevisual indicia corresponding to the location of the treatment, and atleast one of the first image and the second image including the visualindicia.
 14. The method of claim 1, further comprising receiving asignal indicative of a location of an anatomic feature of the anatomicstructure, wherein constructing the three-dimensional model includesadding visual indicia to the three-dimensional model, the visual indiciacorresponding to the location of the anatomic feature, and the firstimage and the second image each including the visual indicia.
 15. Amethod comprising: receiving a signal indicative of a location of amedical device in an anatomic structure of a patient; based at least inpart on the received location signal, updating a three-dimensionalmodel, the three-dimensional model including a three-dimensionalrepresentation of the anatomic structure and a representation of themedical device relative to the anatomic structure; forming a first imageincluding a projection, on a first viewing window of a first imageplane, of at least one portion of the updated three-dimensional model;on a second viewing window of a second image plane, forming a secondimage of another portion of the updated three-dimensional model, theportion of the three-dimensional model projected on the second viewingwindow being less than the at least one portion of the updatedthree-dimensional model projected on the first viewing window, and thesecond image plane intersecting the first image plane; and displaying,on a graphical user interface, the first image and the second image. 16.The method of claim 15, further comprising adjusting the first imageplane and maintaining the second image plane in a fixed orientationrelative to the first image plane.
 17. The method of claim 16, whereinadjusting the first image plane includes orienting the first image planerelative to the received location signal.
 18. The method of claim 17,wherein orienting the first image plane relative to the receivedlocation signal includes orienting the first image plane in a directionperpendicular to a weighted sum of surface normal vectors along aportion, closest to the medical device, of a boundary surface of theanatomic structure in the three-dimensional model.
 19. The method of 15,wherein displaying the first image and the second image includesdisplaying the first image and the second image simultaneously on thegraphical user interface.
 20. The method of claim 15, wherein the secondimage plane is orthogonal to the first image plane.